Broadband passive tracking for augmented reality

ABSTRACT

Technologies are generally described for a broadband passive sensing and tracking system that may employ a number of passive receivers that each have the capability of sensing electromagnetic waves (e.g., Radio Frequency “RF” signals) from surrounding broadcast sources. Each passive receiver may be adapted to sense through one or more antennas. Multiple receivers at different positions may be utilized to form a broadband sensing network adapted to perform collaborative tracking of a scene of interest. According to some examples, a beam-forming algorithm may be applied over the broadband sensing network utilizing an antenna array formed by the passive receivers to localize and track objects.

BACKGROUND

Unless otherwise indicated herein, the materials described in thissection are not prior art to the claims in this application and are notadmitted to be prior art by inclusion in this section.

Augmented reality (AR) refers to a view of a physical (real) worldenvironment whose elements are augmented by virtual, typicallycomputer-generated, imagery, thereby creating a mixed reality. Theaugmentation may be conventionally in real time and in context withenvironmental elements, such a sporting event, a military exercise, agame, etc. AR technology enables the information about surrounding realworld of a person to become interactive and digitally usable by addingobject recognition and image generation. Artificial information aboutthe environment and the objects may be stored and retrieved as aninformation layer separate from a real world view layer.

The present disclosure appreciates that there are several limitationswith AR systems. In supplementing the real world with virtual orcomputer-generated objects that appear to coexist in the same space asthe real world, AR technology allows a user to work with and examinereal three dimensional (3D) objects while visually receiving additionalcomputer-based information about those objects or the task at hand. Inorder to enable users to interact with a mixed virtual and real world ina natural way, an AR system may require knowledge of the user's locationand the position of other objects of interest in the environment throughenvironment sensing. For example, AR systems may need a depth map of thereal scene to support occlusion when rendering. The system may alsoutilize information regarding the object's position and motionparameters, i.e., velocity, acceleration, motion direction, motionpattern, etc. However, various challenges remain with the AR systems inobtaining and processing position and motion parameters.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing and other features of this disclosure will become morefully apparent from the following description and appended claims, takenin conjunction with the accompanying drawings. Understanding that thesedrawings depict only several embodiments in accordance with thedisclosure and are, therefore, not to be considered limiting of itsscope, the disclosure will be described with additional specificity anddetail through use of the accompanying drawings, in which:

FIG. 1 illustrates an example Augmented Reality (AR) system, where someembodiments may be implemented;

FIG. 2 illustrates an example broadband passive tracking systemarchitecture that may provide input to an AR system;

FIG. 3A illustrates a block diagram of example tracking processes byexample AR devices using various transmission sources;

FIG. 3B illustrates a block diagram of tracking operations based oninput to an example AR engine;

FIG. 4 illustrates geometric representation of a passive sensor networkin a system according to embodiments;

FIG. 5 illustrates a general purpose computing device, which may be usedto implement broadband passive tracking in an AR system;

FIG. 6 is a flow diagram illustrating an example method that may beperformed by a computing device such as device 500 in FIG. 5; and

FIG. 7 illustrates a block diagram of an example computer programproduct, all arranged in accordance with at least some embodimentsdescribed herein.

DETAILED DESCRIPTION

In the following detailed description, reference is made to theaccompanying drawings, which form a part hereof. In the drawings,similar symbols typically identify similar components, unless contextdictates otherwise. The illustrative embodiments described in thedetailed description, drawings, and claims are not meant to be limiting.Other embodiments may be utilized, and other changes may be made,without departing from the spirit or scope of the subject matterpresented herein. It will be readily understood that the aspects of thepresent disclosure, as generally described herein, and illustrated inthe Figures, can be arranged, substituted, combined, separated, anddesigned in a wide variety of different configurations, all of which areexplicitly contemplated herein.

This disclosure is generally drawn, inter alia, to methods, apparatus,systems, devices, and/or computer program products related to abroadband passive tracking system for augmented reality applications.

Briefly stated, a broadband passive sensing and tracking systemaccording to some embodiments may include a number of passive receiversthat each have the capability of sensing electromagnetic waves (e.g.Radio Frequency “RF” signals) from surrounding broadcast sources. Eachpassive receiver may be adapted to sense through one or more antennas.Multiple receivers at different positions may be utilized to form abroadband sensing network adapted to perform collaborative tracking of ascene of interest. According to some examples, a beam-forming algorithmmay be applied over the broadband sensing network utilizing an antennaarray formed by the passive receivers to localize and track objects.

FIG. 1 illustrates an example Augmented Reality (AR) system 100,arranged in accordance with at least some embodiments described herein.AR explores the application of computer-generated imagery in live videostreams to expand the real world presentation. Example AR systemsarranged in accordance with the present disclosure may be in controlledenvironments containing a number of sensors and actuators, may includeone or more computing device adapted to process real andcomputer-generated imagery, and may include visualization systems suchas head-mounted displays, virtual retinal displays, monitor or similarregular displays, and comparable devices.

Example AR system 100 includes image sensors 104-1 for capturing liveimages of real scene (objects) 102, as well as tracking sensors 104-2for tracking a position and/or a motion of the objects. Image sensors104-1 may be digital cameras, webcams, or some other image capturingdevices. Tracking sensors 104-2 may include a number of receivingdevices arranged in a passive sensing network to enhance trackingperformance through frequency, bandwidth, and spatial diversity of thenetwork. The receiving devices (e.g., one or more RF receivers) may beadapted to utilize communication signals (e.g., electromagnetic wavessuch as RF signals) from nearby signal sources such as communicationtowers (e.g., cellular telephony communication towers) or communicationbase stations. Tracking sensors 104-2 may be located in differentpositions and may be communicatively coupled to a centralized ordistributed computing system form the collaborative network.

The captured image(s) may be provided to an image processing sub-system106, which may be adapted to perform one or more of digitization ofimages into digital images, receipt of digital images, and/or processingdigital images. Processing of digital images may include one or more ofdetermining locations of feature points in the images, computation ofaffine projections, tracking of edges, filtering, and/or similaroperations. Image processing sub-system 106 may be configured to provideprojection information, such as one or more of the results of the abovedescribed operations, to reality engine 110. Tracking sensors 104-2 maybe configured to provide position and/or motion information associatedwith objects of interest in real scene 102 to reality engine 110.Reality engine 110 may be adapted to execute a graphics process torender scenes based on the captured images that incorporates positionand/or motion information from tracking sensors 104-2. Virtual objectsmay be rendered using the tracking information as discussed in moredetail below.

Image generator 108 may be adapted to receive reference image(s) fromimage sensors 104-1 as well as image data associated with virtualobject(s), and may be adapted to overlay the captured real scene imageswith the image data associated with the virtual object(s) to provide anaugmented scene 114. Display 112 is one example visualization mechanismthat may be utilized in AR system 100. As discussed previously, othertypes of display devices may be used to provide visualization of theaugmented scene 114 for a user of AR system 100.

Image processing sub-system 106, reality engine 110, and image generator108 may be implemented as separate applications, one or more integratedapplications, one or more centralized services, or one or moredistributed services on one more computing devices. Each computingdevice may be either a general purpose computing devices or a specialpurpose computing device that may be a standalone computer, a networkedcomputer system, a general purpose processing unit (e.g., amicro-processor, a micro-controller, a digital signal processor or DSP,etc.), or a special purpose processing unit. If executed on differentcomputing devices, various components of the AR system 100 may beadapted to communicate over one or more networks.

The network(s) may comprise any topology employing servers, clients,switches, routers, modems, Internet service providers (ISPs), and anyappropriate communication media (e.g., wired or wirelesscommunications). A system according to some embodiments may have astatic or dynamic network topology. The network(s) may include a securenetwork such as an enterprise network (e.g., a LAN, WAN, or WLAN), anunsecure network such as a wireless open network (e.g., IEEE 802.11wireless networks), or a world-wide network such (e.g., the Internet).The network(s) may also comprise a plurality of distinct networks thatare adapted to operate together. The network(s) are adapted to providecommunication between the nodes described herein. By way of example, andnot limitation, the network(s) may include wireless media such asacoustic, RF, infrared and other wireless media.

FIG. 2 illustrates an example broadband passive tracking systemarchitecture 200 that may provide input to an AR system. The enhancementin AR technology may include virtual geometric objects superimposed intothe environment or a display of non-geometric information about existingreal objects. Accurate registration is based on accurate tracking of thelocation of a user 230 and sensing other objects in a scene of interest228. An AR system according to at least some embodiments utilizespassive sensing and tracking through Radio Frequency (RF) signals.Passive sensing employs existing illumination sources from nearbytransmission sources (e.g. broadcasting tower 222 or base stations 224and 236) and analyzes received signals scattered from the objectsthrough passive receivers.

Some transmission sources may include, but are not limited to, cellularcommunication sources, or radio frequency (RF) communication sourcessuch as may be used for audio, television or some other datacommunication source Some examples sources may include TV broadcasttowers, Global System for Mobile communications (GSM) towers, CodeDivision Multiple Access (CDMA) cellular communication towers, TimeDivision Multiple Access (TDMA) communication towers, or OrthogonalFrequency-Division Multiple Access (OFDMA) communication towers,Amplitude Modulation (AM) or Frequency Modulation (FM) broadcast towers,Digital Audio Broadcasting (DAB) sources, Digital VideoBroadcasting-Terrestrial (DVB-T) sources, Wireless Local Area Network(WLAN) access points, Wide Area Network (WAN) access points,Metropolitan Area Network (MAN) access points, Personal Area Network(PAN) access points, and comparable ones. Similarly, example AR devicesmay include, but are not limited to, a TV receiver, a digital TVreceiver, a GSM device, a CDMA device, an AM/FM receiver, a DAB device,a DVB-T device, a WLAN device, a WAN device, a MAN device, and a PANdevice, and comparable ones.

The term “broadband”, as used herein, may refer to a wide range offrequencies that can be utilized by receivers of an AR sensing andtracking system according to some embodiments described herein. As such,the term may be distinguished from broadband Internet systems or similartechnologies.

Passive receivers in a broadband network may be a collection of one ormore homogenous sensors that are configured to receive energy from thesame type of illumination source, or heterogeneous sensors that receiveenergy from different types of illumination sources. Passive receiversmay be designed and integrated in AR enabled devices such as handhelddevices or head-worn devices (e.g. AR devices 226, 232, and 234).Passive receivers may be adapted to communicate with one or morecomputing devices through a separate network (e.g. a local area network)to collaboratively carry out passive tracking of the scene of interest228.

Each node in the network, corresponding to a passive receiver, may beadapted to passively receive, capture and/or evaluate signals in the RFsignal spectrum of interest in the environment through either downlinkcommunication channels from nearby base stations or downlinktransmission channels from surrounding broadcasting sources (e.g.broadcasting tower 222 or base stations 224 and 236). The relativemotion between objects and a node in the network can be observed as avariation of the Doppler spectrum of the environment. The motionparameters may be estimated by synthesizing an arbitrary antenna arrayformed by multiple antennas through the network of receivers. Apoint-to-point communication layer of the receiver network may bedeveloped to enable communication and sharing of information between anytwo nodes in the receiver network.

The antenna array may be calibrated adaptively via a location servicesuch as Global Positioning System (GPS) embedded in each passivereceiver by providing accurate, continuous, and/or 3D positioninformation. This information may also be used for compensating for thephase differences between passive receivers due to different locationsrelative to the moving objects as explained in conjunction with FIG. 4below. By updating and sharing the sensing signals over the receivernetwork and the knowledge of the positions of stationary transmissionsources utilized in the receiver network, a beam forming process may beperformed on the synthesized antenna array to obtain the estimates ofmotion parameters and tracking. The beam forming algorithm may bedesigned to adapt to various sensing environments by selecting propernodes in the receiver network and forming an optimal synthesized arrayaccording to the location of receivers.

Each passive receiver may be adapted to operate with two channels, whichmay be denoted as the direct channel and the echo channel. Directchannel signals can correspond to those signals received by the passivereceiver as a result of direct emissions from the surrounding basestations or broadcasting towers, while scattered signals from movingobjects can be fed into the echo channel of the passive receiver. Thesignals may be processed at the receivers and/or at a tracking module,and the resulting tracking information can be provided to AR engine 240for further processing with captured images (video or still) fromimaging sensors (238). The AR engine may be configured to output dataassociated with virtual objects or augmented scenes to other AR devices242 for visualization or further AR related processing. The output datamay also be transmitted back to AR devices 226, 232, 224 forvisualization.

FIG. 3A illustrates a block diagram 300 of example tracking processes byexample AR devices 352, 354, 356 using various transmission sources 344,346, 348, arranged in accordance with at least some embodimentsdescribed herein. As mentioned previously, each passive receiver 361,366, and 372 in a system according to some embodiments may be configuredto receive signals over two channels, where one channel corresponds to adirect path and the other channel corresponds to an echo path (358). Thedirect path is located between transmission sources (e.g. transmissiontowers 344, 346, and 348) and the corresponding AR devices (352, 354,and 356) that incorporate the receivers 361, 366, and 372. The echo pathindicates a communication path for data or other information concerningthe scattered signal from the object of interest 350 to the AR devices352, 354, and 356.

Received signals may be pre-processed at one or more of the individualAR devices 352, 354, and/or 356. The analog pre-processing may includeone or more of signal correction (e.g., orthogonal signal correction,multiplicative signal correction, etc.), bandwidth filtering (e.g.,passive or active filters such as band-pass, low-pass, high-pass, etc.),signal averaging, and comparable processes that may be performed on thedirect channel and echo channel as shown by the AR device processing box360 in block diagram 300. In particular, the received signals from bothchannels may be amplified or attenuated (e.g., gain scaling),down-converted (e.g., de-modulated), and digitized (e.g., quantized by acomparator or an analog-to-digital converter). To mitigate multipathinterference, blind channel equalization may be performed on the directsignal (channel equalization 362, 367, and 373). An adaptive filteringalgorithm may be applied (adaptive filtering 365, 370, 376) on bothchannels after digitization to suppress clutter and to remove the directsignal from the echo channel.

Doppler spectrum for the received signals may be obtained by performinga coherent cross-correlation (363, 368, and 374) between signals fromthe direct channel (i.e., a direct channel signal) and the echo channel(i.e., an echo channel signal). Range compression may be performed onthe cross-correlated baseband signals in range compression filters 364,369, and 375 before the baseband signals are fed (378) through a beamforming process via the receiver network to obtain a parametric space.Tracking objects may be realized by computing the beam forming processoutput of the synthesized antenna array at each time instant at a singlecomputing device or through a distributed process executed by multiplecomputing devices.

FIG. 3B illustrates a block diagram 350 of tracking operations based oninput to an example AR engine, arranged in accordance with at least someembodiments described herein. As discussed above, the baseband signalsmay be fed (378) through a beam forming process in an AR engineprocessing block 380 as shown in diagram 350.

The baseband signal output 382 of one of the AR devices (e.g., AR device352) may be used as a reference to remove phase difference. Phasedifference may occur for scattering center due to different relativelocations of sensor/transmitter pairs. Thus, one of the baseband outputsignals may be used as reference signal to remove the phase differencefrom the other baseband output signal for each AR device pair such thatthe signals are in rotational/angular/phase alignment with one another.After phase compensation 384 in frequency domain, the resulting signalsmay be equivalent to those obtained from the reference transmissiontower but at different sensor locations. This enables targetlocalization and tracking (386) to be performed by the AR engineprocessing module 380 using the phase compensated baseband outputsignals 385 from the AR devices (e.g., in pairs). Thetracking/localization information 387 may then be provided to an ARapplication 388, which may also transmit the information to the ARdevice running the AR client software to enable user feedback,corrective processing, viewing the tracked object's motion, and/orsimilar operations. Mathematical details of possible methods for domainconversion and phase conversation are discussed in detail below inconjunction with FIG. 4.

The diversity gain offered by a broadband passive network in a systemaccording to some embodiments may produce performance gains forenvironment sensing and, thereby, may lead to improved trackingperformance over single-receiver tracking. Diversity may be achieved inseveral aspects.

Frequency and bandwidth diversity result from the use of heterogeneousillumination sources. For example, GSM and DAB communication systems areoperated at different frequencies with different bandwidths fortransmission. Moreover, transmission towers and base stations in thesesystems may also have diverse antenna patterns. The objects illuminatedby these sources generally yield different reflective properties, i.e.,some objects are more visible to specific frequencies.

Spatial diversity can result from different locations of passive sensorsor array elements with respect to the same objects. Likewise, signalsused to identify the objects may have a better path to the object(either transmitted or reflected) from one source versus another. A beamforming algorithm according to at least some embodiments may beconfigured to adaptively select a number and location of trackingsensors in the network to increase the diversity gain to provide anenhanced tracking output. Furthermore, the performance gain may also beachieved from the synthesized antenna array by mitigating multipathfading and by cancelling interfering signals as a result of diversity.

In general, passive sensing employs existing communication links orbroadcast systems, which may potentially reduce the size, weight, andpower consumption of the passive devices to enable AR applications. Inaddition, as the illumination sources for communication and broadcastsystems are ubiquitous, passive receivers are free of additionalfrequency allocation and relatively immune to interference from other RFdevices compared to active receivers.

While embodiments have been discussed above using specific examples,components, algorithms, and configurations, they are intended to providea general guideline to be used for broadband passive tracking in ARsystems. These examples do not constitute a limitation on theembodiments, which may be implemented using other components, modules,algorithms, and configurations using the principles described herein.For example, other wireless technology systems may be utilized to detectscattered signals from objects of interest or a variety of signalprocessing methods may be employed to track objects based on thedirectly received and scattered signals.

FIG. 4 illustrates geometric representation 400 of a passive sensornetwork in a system arranged according to at least some embodimentsdescribed herein. For brevity and simplicity purposes, an object 490with associated location vector r₀ is shown in geometric representation400 with two transmission sources Tx1 and Tx2 at location vectors r_(T1)and r_(T2), respectively. Corresponding receivers Rx1 and Rx2 arelocated at the origin of the Cartesian coordinate system X, Y, Z. If acomplex passive transmit signal denoted as:s _(T)(t)=p(t)e ^((j2πfct))  [1]is transmitted from either transmitter Tx1 or Tx2, the travel distance(492, 494) from each transmitter Txi (i=1, 2) to target scatteringcenter may be expressed as:d _(Ti) =|r _(Ti) −r ₀|,  [2]where p(t) is the baseband signal and f_(c) represents the carrierfrequency.

The travel distance from receivers RXi (i=1, 2) to the target scatteringcenter may be expressed as:d _(xi) =|r ₀|  [3]Under the same scenario, the time delay of the ith channel signal may beexpressed as:

$\begin{matrix}{{{\tau_{i}\left( r_{0} \right)} = {\frac{d_{Ti} - d_{R}}{c} = {\frac{{{r_{Ti} - r_{0}}} - {r_{0}}}{c}\left( {1 \leq i \leq 2} \right)}}},} & \lbrack 4\rbrack\end{matrix}$where c is the velocity of light. The received target echo channelsignal from the ith transmission tower (e.g. Tx1), S_(Ei) (t, r₀) may beexpressed as:S _(Ei)(t,r ₀)=∫_(v) g(r ₀)p _(i)(t−τ _(i)(r _(o)))exp(j2πf _(ci)(t−τ_(i)(r _(o))))dr ₀,  [5]where (1≦i≦2). The direct channel return, S_(Di)(t), for the samescenario may be expressed as:S _(Di)(t)=pi(t−τ _(di))exp(j2πf _(ci)(t−τ _(di))),  [6]where τ_(di) is the direct path travel time, which is equivalent toτ_(di)=|r_(Ti)|/c. g(r₀) is the target reflectivity function.Integration may be taken over the entire spatial area of object scene ofinterest V (496).

After direct and target echo channel processing, the received signal maybe rewritten as:S _(i)(t,r ₀)=∫_(v) g(r ₀)p _(i) ^(c)(t−τ _(i) ^(c)(r _(o)))exp(j2πf_(ci)(t−τ _(i) ^(c)(r _(o))))dr ₀,  [7]where

$\begin{matrix}{{\tau_{i}^{c}\left( r_{0} \right)} = {{{\tau_{i}\left( r_{0} \right)} - \tau_{di}} = \frac{{{r_{Ti} - r_{0}}} + {r_{0}} - {r_{Ti}}}{c}}} & \lbrack 8\rbrack\end{matrix}$and p_(i) ^(c)(t) is the compensated baseband signal for eachtransmitter (i=1, 2).

After baseband conversion and range compression, the received signals infrequency domain may be written as:S ₁(f)≈rec(f/B ₁)∫_(v) g(r ₀)exp(−j2π(f+f _(c1))τ₁ ^(c)(r _(o)))dr ₀,and  [9]S ₂(f)≈rec(f/B ₂)∫_(v) g(r ₀)exp(−j2π(f+f _(c2))τ₂ ^(c)(r _(o)))dr₀,  [10]where B₁ and B₂ are bandwidths corresponding to transmission sources Tx1and Tx2, respectively. Assuming the difference between centerfrequencies of the transmitters is Δf₂₁=f_(c2)−f_(c1), the receivedsignal for the second transmitter may be shifted in the frequency domainby Δf₂₁ and rewritten as:S′ ₂(f)=S ₂(f−Δf ₂₁)=rec((f−Δf ₂₁)/B ₂)∫_(v) g(r ₀)exp(−j2π(f+f _(c1))τ₂^(c)(r _(o)))dr ₀.  [11]

A phase difference between the signals occurs for the scattering centerdue to the different relative location of sensor/transmitter pairs.Consequently, the phase difference for object 490 may be expressed as:Δφ(r ₀)=−2π(f−f _(c1))ε(r ₀),  [12]where

${ɛ\left( r_{0} \right)} = {{{\tau_{2}^{c}\left( r_{0} \right)} - {\tau_{1}^{c}\left( r_{0} \right)}} = {\frac{\left( {{{r_{T\; 2} - r_{0}}} - {{r_{T\; 1} - r_{0}}}} \right) - \left( {{r_{T\; 2}} - {r_{T\; 1}}} \right)}{c}.}}$

After compensating for the phase difference, Δφ(r₀), the two spectra maybe equivalent to those obtained from the first transmission tower but atdifferent sensor locations. To enable centralized target localizationand tracking, the phase difference between any two sensor/transmitterpairs needs to be compensated. After phase compensation, the resultingsignals may be equivalent to those obtained from the referencetransmission tower but at different sensor locations.

FIG. 5 illustrates a general purpose computing device 500, which may beused to implement broadband passive tracking in an AR system arrangedaccording to at least some embodiments of the present disclosure. In avery basic configuration 502, computing device 500 typically includesone or more processors 504 and a system memory 506. A memory bus 508 maybe used for communicating between processor 504 and system memory 506.

Depending on the desired configuration, processor 504 may be of any typeincluding but not limited to a microprocessor (μP), a microcontroller(μC), a digital signal processor (DSP), or any combination thereof.Processor 504 may include one more levels of caching, such as a levelcache memory 512, a processor core 514, and registers 516. Exampleprocessor core 514 may include an arithmetic logic unit (ALU), afloating point unit (FPU), a digital signal processing core (DSP Core),or any combination thereof. An example memory controller 518 may also beused with processor 504, or in some implementations memory controller518 may be an internal part of processor 504.

Depending on the desired configuration, system memory 506 may be of anytype including but not limited to volatile memory (such as RAM),non-volatile memory (such as ROM, flash memory, etc.) or any combinationthereof. System memory 506 may include an operating system 520, one ormore applications 522, and program data 524. Application 522 may includean AR engine 526 that is arranged to adjust operational parameters of anobject recognition and modeling system in conjunction with trackinginformation about the objects provided from a network of passivereceivers as discussed above. Program data 524 may include one or moreof imaging data 528-1, tracking data 528-2, and similar data asdiscussed above in conjunction with FIGS. 3A and 3B. This data may beuseful in generating virtual objects to be augmented onto a real scenebased on position and/or motion information associated with the objectsin the real scene as is described herein. In some embodiments,application 522 may be arranged to operate with program data 524 onoperating system 520 such that three dimensional objects are tracked andvirtual representations generated as described herein. This describedbasic configuration 502 is illustrated in FIG. 5 by those componentswithin the inner dashed line.

Computing device 500 may have additional features or functionality, andadditional interfaces to facilitate communications between basicconfiguration 502 and any required devices and interfaces. For example,a bus/interface controller 530 may be used to facilitate communicationsbetween basic configuration 502 and one or more data storage devices 532via a storage interface bus 534. Data storage devices 532 may beremovable storage devices 536, non-removable storage devices 538, or acombination thereof. Examples of removable storage and non-removablestorage devices include magnetic disk devices such as flexible diskdrives and hard-disk drives (HDD), optical disk drives such as compactdisk (CD) drives or digital versatile disk (DVD) drives, solid statedrives (SSD), and tape drives to name a few. Example computer storagemedia may include volatile and nonvolatile, removable and non-removablemedia implemented in any method or technology for storage ofinformation, such as computer readable instructions, data structures,program modules, or other data.

System memory 506, removable storage devices 536 and non-removablestorage devices 538 are examples of computer storage media. Computerstorage media includes, but is not limited to, RAM, ROM, EEPROM, flashmemory or other memory technology, CD-ROM, digital versatile disks (DVD)or other optical storage, magnetic cassettes, magnetic tape, magneticdisk storage or other magnetic storage devices, or any other mediumwhich may be used to store the desired information and which may beaccessed by computing device 500. Any such computer storage media may bepart of computing device 500.

Computing device 500 may also include an interface bus 540 forfacilitating communication from various interface devices (e.g., outputdevices 542, peripheral interfaces 544, and communication devices 546)to basic configuration 502 via bus/interface controller 530. Exampleoutput devices 542 include a graphics processing unit 548 and an audioprocessing unit 550, which may be configured to communicate to variousexternal devices such as a display or speakers via one or more A/V ports552. Example peripheral interfaces 544 include a serial interfacecontroller 554 or a parallel interface controller 556, which may beconfigured to communicate with external devices such as input devices(e.g., keyboard, mouse, pen, voice input device, touch input device,etc.) or other peripheral devices (e.g., printer, scanner, etc.) via oneor more I/O ports 558. An example communication device 546 includes anetwork controller 560, which may be arranged to facilitatecommunications with one or more other computing devices 562 over anetwork communication link via one or more communication ports 564.

The network communication link may be one example of a communicationmedia. Communication media may typically be embodied by computerreadable instructions, data structures, program modules, or other datain a modulated data signal, such as a carrier wave or other transportmechanism, and may include any information delivery media. A “modulateddata signal” may be a signal that has one or more of its characteristicsset or changed in such a manner as to encode information in the signal.By way of example, and not limitation, communication media may includewired media such as a wired network or direct-wired connection, andwireless media such as acoustic, radio frequency (RF), microwave,infrared (IR) and other wireless media. The term computer readable mediaas used herein may include both storage media and communication media.

Computing device 500 may be implemented as a portion of a small-formfactor portable (or mobile) electronic device such as a cell phone, apersonal data assistant (PDA), a personal media player device, awireless web-watch device, a personal headset device, an applicationspecific device, or a hybrid device that include any of the abovefunctions. Computing device 500 may also be implemented as a personalcomputer including both laptop computer and non-laptop computerconfigurations. Moreover computing device 500 may be implemented as anetworked system or as part of a general purpose or specialized server.

Example embodiments may also include methods. These methods can beimplemented in any number of ways, including the structures describedherein. One such way is by machine operations, of devices of the typedescribed in the present disclosure. Another optional way is for one ormore of the individual operations of the methods to be performed inconjunction with one or more human operators performing some of theoperations while other operations are performed by machines. These humanoperators need not be collocated with each other, but each can be onlywith a machine that performs a portion of the program. In otherexamples, the human interaction can be automated such as by pre-selectedcriteria that are machine automated.

FIG. 6 is a flow diagram illustrating an example method that may beperformed by a computing device, such as device 500 in FIG. 5, arrangedin accordance with at least some embodiments described herein. Theoperations described in blocks 622 through 630 may be stored ascomputer-executable instructions in a computer-readable medium such ascomputer-readable medium 620 of computing device 610.

A process of employing a broadband passive tracking system for augmentedreality may begin with operation 622, “SENSE SIGNAL”. At operation 622,the direct and the echo signals are sensed by a network of receiverssuch as receivers 361 of FIG. 3A. The receivers may be integrated intowireless devices such as AR devices 352, 354, and 356. The directsignals may be received from a variety of transmission sources such ascellular towers, TV broadcast towers, and the like, while the echosignals are scattered from one or more objects of interest. The receivedsignals may be pre-processed as discussed in conjunction with FIG. 3A.

Operation 622 may be followed by operation 624, “CROSS-CORRELATE.” Atoperation 624, the received (and pre-processed) signals may becross-correlated to obtain Doppler spectrum for the received signals.Cross-correlation may also be performed in the individual AR devices352, 354, and 356 of FIG. 3A.

Operation 624 may be followed by operation 626, “PHASE COMPENSATE.” Atoperation 626, the received signals may be phase compensated using thesignal from one of the AR devices (e.g., baseband output signal 382 fromAR device 352) at a central tracking/localization module such astracking/localization module 386 of FIG. 3B. Phase difference may occurfor scattering center due to different relative locations ofsensor/transmitter pairs. Thus, one of the baseband output signals maybe used as reference signal to remove the phase difference from theother baseband output signal for each AR device pair such that thesignals are in rotational/angular/phase alignment with one another.After phase compensation 384 in frequency domain, the resulting signals385 may be equivalent to those obtained from the reference transmissiontower but at different sensor locations.

Operation 626 may be followed by operation 628, “DETERMINEPOSITION/MOTION.” At operation 628, the phase compensated signals may befed through a beam forming process at an AR engine such as AR engine 240of FIG. 2 via the receiver network to obtain a parametric space.Tracking (determining position and/or motion parameters) objects may berealized by computing the beam forming process output of the synthesizedantenna array at each time instant.

Operation 628 may be followed by operation 630, “PROVIDE POSITION/MOTIONINFORMATION To AR APPLICATION.” At operation 630, the tracking(position/motion) information may be provided to an AR application foruse in rendering an augmented reality scene superimposing virtualobjects with real scene image(s). The AR scene may be visualized througha visualization device such as those discussed in FIG. 1.

The operations included in the above described process are forillustration purposes. A broadband passive tracking system for obtainingposition and/or motion information associated with objects in an ARsystem may be implemented by similar processes with fewer or additionaloperations. In some examples, the operations may be performed in adifferent order. In some other examples, various operations may beeliminated. In still other examples, various operations may be dividedinto additional operations, or combined together into fewer operations.

FIG. 7 illustrates a block diagram of an example computer programproduct 700 arranged in accordance with at least some embodimentsdescribed herein. In some examples, as shown in FIG. 7, computer programproduct 700 may include a signal bearing medium 702 that may alsoinclude machine readable instructions 704 that, when executed by, forexample, a processor, may provide the functionality described above withrespect to FIG. 5 and FIG. 6. Thus, for example, referring to processor504, the AR engine 526 may undertake one or more of the tasks shown inFIG. 7 in response to instructions 704 conveyed to processor 504 bymedium 702 to perform actions associated with object recognition basedon dynamic modeling as described herein. Some of those instructions mayinclude sensing signal(s), cross-correlating signal(s), determiningrange of object(s), determining position/motion information, andproviding the position/motion information to an AR engine.

In some implementations, signal bearing medium 702 depicted in FIG. 7may encompass a computer-readable medium 706, such as, but not limitedto, a hard disk drive, a Compact Disc (CD), a Digital Video Disk (DVD),a digital tape, memory, etc. In some implementations, signal bearingmedium 702 may encompass a recordable medium 708, such as, but notlimited to, memory, read/write (R/W) CDs, R/W DVDs, etc. In someimplementations, signal bearing medium 702 may encompass acommunications medium 710, such as, but not limited to, a digital and/oran analog communication medium (e.g., a fiber optic cable, a waveguide,a wired communications link, a wireless communication link, etc.). Thus,for example, program product 700 may be conveyed to one or more modulesof the processor 504 by an RF signal bearing medium 702, where thesignal bearing medium 702 is conveyed by a wireless communicationsmedium 710 (e.g., a wireless communications medium conforming with theIEEE 802.11 standard).

There is little distinction left between hardware and softwareimplementations of aspects of systems; the use of hardware or softwareis generally (but not always, in that in certain contexts the choicebetween hardware and software may become significant) a design choicerepresenting cost vs. efficiency tradeoffs. There are various vehiclesby which processes and/or systems and/or other technologies describedherein may be effected (e.g., hardware, software, and/or firmware), andthat the preferred vehicle will vary with the context in which theprocesses and/or systems and/or other technologies are deployed. Forexample, if an implementer determines that speed and accuracy areparamount, the implementer may opt for a mainly hardware and/or firmwarevehicle; if flexibility is paramount, the implementer may opt for amainly software implementation; or, yet again alternatively, theimplementer may opt for some combination of hardware, software, and/orfirmware.

The foregoing detailed description has set forth various embodiments ofthe devices and/or processes via the use of block diagrams, flowcharts,and/or examples. Insofar as such block diagrams, flowcharts, and/orexamples contain one or more functions and/or operations, it will beunderstood by those within the art that each function and/or operationwithin such block diagrams, flowcharts, or examples may be implemented,individually and/or collectively, by a wide range of hardware, software,firmware, or virtually any combination thereof. In one embodiment,several portions of the subject matter described herein may beimplemented via Application Specific Integrated Circuits (ASICs), FieldProgrammable Gate Arrays (FPGAs), digital signal processors (DSPs), orother integrated formats. However, those skilled in the art willrecognize that some aspects of the embodiments disclosed herein, inwhole or in part, may be equivalently implemented in integratedcircuits, as one or more computer programs running on one or morecomputers (e.g., as one or more programs running on one or more computersystems), as one or more programs running on one or more processors(e.g. as one or more programs running on one or more microprocessors),as firmware, or as virtually any combination thereof, and that designingthe circuitry and/or writing the code for the software and or firmwarewould be well within the skill of one of skill in the art in light ofthis disclosure.

The present disclosure is not to be limited in terms of the particularembodiments described in this application, which are intended asillustrations of various aspects. Many modifications and variations canbe made without departing from its spirit and scope, as will be apparentto those skilled in the art. Functionally equivalent methods andapparatuses within the scope of the disclosure, in addition to thoseenumerated herein, will be apparent to those skilled in the art from theforegoing descriptions. Such modifications and variations are intendedto fall within the scope of the appended claims. The present disclosureis to be limited only by the terms of the appended claims, along withthe full scope of equivalents to which such claims are entitled. It isto be understood that this disclosure is not limited to particularmethods, reagents, compounds compositions or biological systems, whichcan, of course, vary. It is also to be understood that the terminologyused herein is for the purpose of describing particular embodimentsonly, and is not intended to be limiting.

In addition, those skilled in the art will appreciate that themechanisms of the subject matter described herein are capable of beingdistributed as a program product in a variety of forms, and that anillustrative embodiment of the subject matter described herein appliesregardless of the particular type of signal bearing medium used toactually carry out the distribution. Examples of a signal bearing mediuminclude, but are not limited to, the following: a recordable type mediumsuch as a floppy disk, a hard disk drive, a Compact Disc (CD), a DigitalVideo Disk (DVD), a digital tape, a computer memory, etc.; and atransmission type medium such as a digital and/or an analogcommunication medium (e.g., a fiber optic cable, a waveguide, a wiredcommunications link, a wireless communication link, etc.).

Those skilled in the art will recognize that it is common within the artto describe devices and/or processes in the fashion set forth herein,and thereafter use engineering practices to integrate such describeddevices and/or processes into data processing systems. That is, at leasta portion of the devices and/or processes described herein may beintegrated into a data processing system via a reasonable amount ofexperimentation. Those having skill in the art will recognize that atypical data processing system generally includes one or more of asystem unit housing, a video display device, a memory such as volatileand non-volatile memory, processors such as microprocessors and digitalsignal processors, computational entities such as operating systems,drivers, graphical user interfaces, and applications programs, one ormore interaction devices, such as a touch pad or screen, and/or controlsystems including feedback loops and control motors (e.g., feedback forsensing position and/or velocity of gantry systems; control motors formoving and/or adjusting components and/or quantities).

A typical data processing system may be implemented utilizing anysuitable commercially available components, such as those typicallyfound in data computing/communication and/or networkcomputing/communication systems. The herein described subject mattersometimes illustrates different components contained within, orconnected with, different other components. It is to be understood thatsuch depicted architectures are merely exemplary, and that in fact manyother architectures may be implemented which achieve the samefunctionality. In a conceptual sense, any arrangement of components toachieve the same functionality is effectively “associated” such that thedesired functionality is achieved. Hence, any two components hereincombined to achieve a particular functionality may be seen as“associated with” each other such that the desired functionality isachieved, irrespective of architectures or intermediate components.Likewise, any two components so associated may also be viewed as being“operably connected”, or “operably coupled”, to each other to achievethe desired functionality, and any two components capable of being soassociated may also be viewed as being “operably couplable”, to eachother to achieve the desired functionality. Specific examples ofoperably couplable include but are not limited to physically connectableand/or physically interacting components and/or wirelessly interactableand/or wirelessly interacting components and/or logically interactingand/or logically interactable components.

With respect to the use of substantially any plural and/or singularterms herein, those having skill in the art can translate from theplural to the singular and/or from the singular to the plural as isappropriate to the context and/or application. The varioussingular/plural permutations may be expressly set forth herein for sakeof clarity.

It will be understood by those within the art that, in general, termsused herein, and especially in the appended claims (e.g., bodies of theappended claims) are generally intended as “open” terms (e.g., the term“including” should be interpreted as “including but not limited to,” theterm “having” should be interpreted as “having at least,” the term“includes” should be interpreted as “includes but is not limited to,”etc.). It will be further understood by those within the art that if aspecific number of an introduced claim recitation is intended, such anintent will be explicitly recited in the claim, and in the absence ofsuch recitation no such intent is present. For example, as an aid tounderstanding, the following appended claims may contain usage of theintroductory phrases “at least one” and “one or more” to introduce claimrecitations. However, the use of such phrases should not be construed toimply that the introduction of a claim recitation by the indefinitearticles “a” or “an” limits any particular claim containing suchintroduced claim recitation to embodiments containing only one suchrecitation, even when the same claim includes the introductory phrases“one or more” or “at least one” and indefinite articles such as “a” or“an” (e.g., “a” and/or “an” should be interpreted to mean “at least one”or “one or more”); the same holds true for the use of definite articlesused to introduce claim recitations. In addition, even if a specificnumber of an introduced claim recitation is explicitly recited, thoseskilled in the art will recognize that such recitation should beinterpreted to mean at least the recited number (e.g., the barerecitation of “two recitations,” without other modifiers, means at leasttwo recitations, or two or more recitations).

Furthermore, in those instances where a convention analogous to “atleast one of A, B, and C, etc.” is used, in general such a constructionis intended in the sense one having skill in the art would understandthe convention (e.g., “a system having at least one of A, B, and C”would include but not be limited to systems that have A alone, B alone,C alone, A and B together, A and C together, B and C together, and/or A,B, and C together, etc.). In those instances where a conventionanalogous to “at least one of A, B, or C, etc.” is used, in general sucha construction is intended in the sense one having skill in the artwould understand the convention (e.g., “a system having at least one ofA, B, or C” would include but not be limited to systems that have Aalone, B alone, C alone, A and B together, A and C together, B and Ctogether, and/or A, B, and C together, etc.). It will be furtherunderstood by those within the art that virtually any disjunctive wordand/or phrase presenting two or more alternative terms, whether in thedescription, claims, or drawings, should be understood to contemplatethe possibilities of including one of the terms, either of the terms, orboth terms. For example, the phrase “A or B” will be understood toinclude the possibilities of “A” or “B” or “A and B.”

In addition, where features or aspects of the disclosure are describedin terms of Markush groups, those skilled in the art will recognize thatthe disclosure is also thereby described in terms of any individualmember or subgroup of members of the Markush group.

As will be understood by one skilled in the art, for any and allpurposes, such as in terms of providing a written description, allranges disclosed herein also encompass any and all possible subrangesand combinations of subranges thereof. Any listed range can be easilyrecognized as sufficiently describing and enabling the same range beingbroken down into at least equal halves, thirds, quarters, fifths,tenths, etc. As a non-limiting example, each range discussed herein canbe readily broken down into a lower third, middle third and upper third,etc. As will also be understood by one skilled in the art all languagesuch as “up to,” “at least,” “greater than,” “less than,” and the likeinclude the number recited and refer to ranges which can be subsequentlybroken down into subranges as discussed above. Finally, as will beunderstood by one skilled in the art, a range includes each individualmember. Thus, for example, a group having 1-3 cells refers to groupshaving 1, 2, or 3 cells. Similarly, a group having 1-5 cells refers togroups having 1, 2, 3, 4, or 5 cells, and so forth.

While various aspects and embodiments have been disclosed herein, otheraspects and embodiments will be apparent to those skilled in the art.The various aspects and embodiments disclosed herein are for purposes ofillustration and are not intended to be limiting, with the true scopeand spirit being indicated by the following claims.

What is claimed is:
 1. A method to track one or more objects of interestin an Augmented Reality (AR) system, the method comprising: receivingsignals at each one of a plurality of tracking sensors, wherein each ofthe plurality of tracking sensors form part of a passive broadbandsensing network, the passive broadband sensing network includes aplurality of passive receivers arranged at different positions in ascene and adapted to perform collaborative tracking of the scene bysensing signals from a plurality of transmission sources, the receivedsignals correspond to one or more of a direct signal and/or an echosignal, each echo signal is associated with a corresponding one of thedirect signals that is scattered by an object of interest, and thedirect signals are transmitted from transmission sources that includeone or more of: TV broadcast towers, Global System for Mobilecommunications (GSM) towers, Code Division Multiple Access (CDMA)cellular communication towers, Amplitude Modulation (AM) or FrequencyModulation (FM) broadcast towers, Digital Audio Broadcasting (DAB)sources, Digital Video Broadcasting-Terrestrial (DVB-T) sources,Wireless Local Area Network (WLAN) access points, Wide Area Network(WAN) access points, Metropolitan Area Network (MAN) access points,and/or Personal Area Network (PAN) access points; processing thereceived signals to generate data associated with the object of interestthrough one or more of determining a location of one or more featurepoints in an image, computation of affine projections, tracking ofedges, and filtering, and wherein the data corresponds to one or more ofa position parameter and/or a motion parameter associated with theobject of interest; developing a point-to-point communication layer toenable communication and sharing of information between any two sensorsin the passive broadband sensing network; and augmenting the scene thatincludes the object of interest, by employing one or more of theposition parameter and/or the motion parameter to augment the object ofinterest in the scene with at least one virtual object.
 2. The methodaccording to claim 1, wherein the signals are received at the pluralityof tracking sensors integrated into a plurality of wirelesscommunication devices communicating with the plurality of transmissionsources.
 3. The method according to claim 2, wherein the receivedsignals correspond to one or more of: a TV signal, a digital TV signal,a GSM signal, a CDMA signal, an AM/FM signal, a DAB signal, a DVB-Tsignal, a WLAN signal, a WAN signal, a MAN signal, and/or a PAN signal.4. The method according to claim 1, wherein processing the receivedsignals includes one or more of pre-processing the received signals;identifying each of the received signals as one or more of the directsignal and/or the echo signal through the filtering, which includesadaptive filtering; deriving Doppler spectra of the direct signal andthe echo signal; and/or phase compensating the Doppler spectra of theecho signals.
 5. The method according to claim 4, wherein pre-processingthe received signals includes one or more of signal correction,bandwidth correction, signal averaging, amplification, down-conversion,and/or digitization.
 6. The method according to claim 4, whereinprocessing the received signals to generate data associated with theobject of interest comprises employing a beam forming process on thephase compensated signals to determine one or more of position and/ormotion parameters associated with the object of interest.
 7. The methodaccording to claim 6, wherein employing the beam forming process furthercomprises adaptively selecting a number and a location of the trackingsensors.
 8. The method according to claim 1, wherein the signals arereceived from the tracking sensors that are communicatively coupledthrough a wireless network distinct from a communication network ofcorresponding wireless communication devices.
 9. The method according toclaim 8, further comprising dynamically configuring the wireless networkof the tracking sensors to mitigate multipath fading and reduceinterference through diversity of antennas of the wireless communicationdevices.
 10. A system to track an object of interest in a real scene inAugmented Reality (AR) systems, the system comprising: a plurality oftracking sensors integrated into a plurality of handheld or head-wornwireless communication devices in communication with a plurality oftransmission sources as part of a passive broadband sensing network thatincludes a plurality of passive receivers arranged at differentpositions adapted to perform collaborative tracking of the scene bysensing signals from the plurality of transmission sources, eachtracking sensor adapted to: receive, through downlink communicationchannels, signals corresponding to one or more of a direct signal and/oran echo signal, wherein the direct signals are associated with one ormore corresponding transmission sources that include one or more of: TVbroadcast towers, Global System for Mobile communications (GSM) towers,Code Division Multiple Access (CDMA) cellular communication towers,Amplitude Modulation (AM) or Frequency Modulation (FM) broadcast towers,Digital Audio Broadcasting (DAB) sources, Digital VideoBroadcasting-Terrestrial (DVB-T) sources, Wireless Local Area Network(WLAN) access points, Wide Area Network (WAN) access points,Metropolitan Area Network (MAN) access points, and/or Personal AreaNetwork (PAN) access points, and wherein the echo signals are associatedwith the one or more direct signals scattered from the object ofinterest; and pre-process the received signals at one or more of thewireless communication devices with active and passive bandwidthfilters; an image processing server adapted to derive image informationbased on a captured 2D image associated with the object of interest; areality server adapted to: enable communication and information sharingbetween any two tracking sensors in the passive broadband sensingnetwork through a point-to-point communication layer; communicate withthe plurality of tracking sensors through a wireless network; receivethe pre-processed signals; track the object of interest based on thepre-processed signals; and augment the real scene based on the trackedobject of interest and the image information.
 11. The system accordingto claim 10, wherein each tracking sensor comprises: a blind channelequalizer adapted to process received direct signals to mitigatemultipath interference; an adaptive filter adapted to process receivedecho signals to suppress clutter; and a correlator adapted tocross-correlate each direct signal and each echo signal to generate aDoppler spectrum signal.
 12. The system according to claim 11, whereinone or more functions of the blind channel equalizer, the adaptivefilter, and/or the correlator are provided by a digital signalprocessor.
 13. The system according to claim 10, wherein the realityserver comprises: a phase compensation module adapted to phasecompensate the pre-processed signals received from the tracking sensor;a tracking/localization module adapted to determine one or moreof-position and/or motion parameters for the object of interest based onthe phase compensated signals; a reality engine adapted to generate oneor more virtual objects based on position/motion information of theobject of interest and the image information; and an image generatoradapted to augment the real scene with the one or more virtual objects.14. The system according to claim 10, further comprising a visualizationdevice adapted to generate a visualization of the augmented real scene.15. The system according to claim 14, wherein the visualization devicecomprises one or more of a head-mounted display, a virtual display,and/or a monitor.
 16. The system according to claim 10, wherein aportion of the tracking sensors include homogeneous sensors adapted toreceive a same type transmission and another portion of the trackingsensors include heterogeneous sensors adapted to receive different typesof transmissions.
 17. The system according to claim 10, wherein thereality server is further adapted to calibrate a synthesized array ofantennas, the synthesized array of antennas comprising antennas fromeach of the tracking sensors, each of the tracking sensors being adaptedto employ a location based service.
 18. An apparatus to track one ormore objects of interest in an Augmented Reality (AR) system, theapparatus comprising: a wireless communication device; and an antennaand a tracking sensor included in the wireless communication device,wherein the wireless communication device is adapted to: receive signalswith the antenna, wherein the received signals correspond to one or moreof direct signals from one or more transmission sources or echo signalsthat are scattered from one or more of the objects of interest as aresult of one or more of the direct signals, the direct signalstransmitted from the one or more transmission sources that include oneor more of: TV broadcast towers, Global System for Mobile communications(GSM) towers, Code Division Multiple Access (CDMA) cellularcommunication towers, Amplitude Modulation (AM) or Frequency Modulation(FM) broadcast towers, Digital Audio Broadcasting (DAB) sources, DigitalVideo Broadcasting-Terrestrial (DVB-T) sources, Wireless Local AreaNetwork (WLAN) access points, Wide Area Network (WAN) access points,Metropolitan Area Network (MAN) access points, and/or Personal AreaNetwork (PAN) access points; pre-process the received signals to obtainone or more frequency domain signals; cross-correlate each direct signaland each echo signal to generate a Doppler spectrum signal; performrange compression on the cross-correlated signals in a range compressionfilter; feed the cross-correlated signals through a beam forming processto obtain a parametric space to derive location and motion information;phase compensate the one or more frequency domain signals to generatephase compensated signals; enable communication and information sharingbetween any two sensors in the passive broadband sensing network througha point-to-point communication layer, wherein the tracking sensorincluded in the wireless communication device forms the passivebroadband sensing network with other tracking sensors included in otherwireless communication devices; derive the location and motioninformation associated with one or more of the objects of interest fromthe phase compensated signals; and augment a real scene that includesthe one or more objects of interest, by employing the location andmotion parameters to augment the one or more objects of interest in thereal scene with at least one virtual object.
 19. The apparatus accordingto claim 18, wherein the wireless communication device is adapted topre-process the received signals by application of one or more of signalcorrection, bandwidth correction, signal averaging, amplification,down-conversion, and/or digitization.
 20. The apparatus according toclaim 18, wherein the wireless communication device includes one or moreof: a TV receiver, a digital TV receiver, a GSM device, a CDMA device,an AM/FM receiver, a DAB device, a DVB-T device, a WLAN device, a WANdevice, a MAN device, and/or a PAN device.
 21. An apparatus to track oneor more objects of interest in Augmented Reality (AR) systems, theapparatus comprising: a memory; a processor coupled to the memory, theprocessor configured in cooperation with the memory to execute an ARengine such that the processor is adapted to perform or cause to beperformed: communicate with a plurality of tracking sensors integratedinto handheld and head-worn AR enabled wireless devices through awireless network that is distinct from communication networks ofcorresponding AR enabled wireless devices; adaptively select a numberand a location of the plurality of tracking sensors, wherein theplurality of tracking sensors form a passive broadband sensing networkthat includes a plurality of passive receivers arranged at differentpositions and is adapted to perform collaborative tracking of a realscene by sensing signals from a plurality of transmission sources;receive, through downlink communication channels, pre-processed signalsthat are detected by the plurality of tracking sensors, wherein thepre-processed signals correspond to one or more of direct signals and/orecho signals that result from the direct signals being reflected by theone or more objects of interest, and wherein the direct signals aretransmitted from the plurality of transmission sources that include oneor more of: TV broadcast towers, Global System for Mobile communications(GSM) towers, Code Division Multiple Access (CDMA) cellularcommunication towers, Amplitude Modulation (AM) or Frequency Modulation(FM) broadcast towers, Digital Audio Broadcasting (DAB) sources, DigitalVideo Broadcasting-Terrestrial (DVB-T) sources, Wireless Local AreaNetwork (WLAN) access points, Wide Area Network (WAN) access points,Metropolitan Area Network (MAN) access points, and/or Personal AreaNetwork (PAN) access points; cross-correlate each direct signal and eachecho signal to generate a Doppler spectrum signal; perform rangecompression on the cross-correlated signals in a range compressionfilter; apply a beam forming process to the received cross-correlatedsignals to obtain a parametric space to estimate location and motionparameters for one or more of the objects of interest based on theparametric space; develop a point-to-point communication layer to enablecommunication and information sharing between any two sensors in thepassive broadband sensing network; and augment the real scene thatincludes the one or more objects of interest, by employing the locationand motion parameters to augment the one or more objects of interest inthe real scene with at least one virtual object.
 22. The apparatusaccording to claim 21, wherein the processor is further configured bythe beam forming process to select tracking sensors associated with oneor more of the plurality of transmission sources having distinctfrequencies, transmission sources having distinct bandwidths, and/ortransmission sources having distinct antenna patterns, so that diversitygain is increased.
 23. The apparatus according to claim 21, wherein theprocessor is further adapted by the AR engine to provide the estimatedlocation and motion parameters for the one or more objects of interestto an AR application, wherein the AR application is configured togenerate an augmented real scene by superimposition of virtual objectsgenerated based on the estimated location and motion parameters for atleast one of the objects of interest and a digitized image of at leastone of the objects of interest.
 24. A computer-readable memory devicehaving instructions stored thereon to track one or more objects ofinterest in Augmented Reality (AR) systems, the instructions comprising:at a tracking sensor included in a handheld or head-worn wirelessdevice, detecting one or more of direct signals received from one ormore transmission sources or echo signals that are scattered from one ormore of the objects of interest as a result of one or more of the directsignals, the direct signals transmitted from the one or moretransmission sources that include one or more of: TV broadcast towers,Global System for Mobile communications (GSM) towers, Code DivisionMultiple Access (CDMA) cellular communication towers, AmplitudeModulation (AM) or Frequency Modulation (FM) broadcast towers, DigitalAudio Broadcasting (DAB) sources, Digital Video Broadcasting-Terrestrial(DVB-T) sources, Wireless Local Area Network (WLAN) access points, WideArea Network (WAN) access points, Metropolitan Area Network (MAN) accesspoints, and/or Personal Area Network (PAN) access points; pre-processingthe received signals at the tracking sensor, wherein pre-processing thereceived signals includes determining a location of one or more featurepoints in an image, computation of affine projections, tracking ofedges, and active and passive bandwidth filtering; enablingcommunication and information sharing between any two sensors in thepassive broadband sensing network through a point-to-point communicationlayer, wherein the tracking sensor included in the wireless device formsthe passive broadband sensing network with other tracking sensorsincluded in other wireless devices; deriving a frequency spectrum signalbased on cross-correlating each pre-processed direct and echo signal;performing range compression on the cross-correlated signals in a rangecompression filter; applying a beam forming process to the receivedcross-correlated signals to obtain a parametric space to estimatelocation and motion parameters for one or more of the objects ofinterest; phase compensating pairs of frequency spectrum signalsreceived from a plurality of tracking sensors at a phase-compensationmodule; deriving location and/or motion information associated with oneor more of the objects of interest from the phase compensated signals ata tracking/localization module by adaptively selecting a number and alocation of the plurality of tracking sensors; and augmenting a realscene that includes the one or more objects of interest, by employingthe derived location and motion information to augment the one or moreobjects of interest in the real scene with at least one virtual object.25. The computer-readable memory device of claim 24, wherein theinstructions further comprise: applying blind channel equalization tothe received direct signals to mitigate multipath interference; andadaptively filtering the received echo signals to suppress clutter. 26.The computer-readable memory device of claim 24, wherein theinstructions further comprise: transmitting the location and/or motioninformation to a wireless device executing an AR client application. 27.The computer-readable memory device of claim 24, wherein theinstructions further comprise: dynamically configuring a network of thewireless devices to mitigate multipath fading and reduce interferencethrough diversity of antennas of the wireless devices.